{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"NLU_training_multi_lingual_multi_class_text_classifier_demo_amazon.ipynb","provenance":[],"collapsed_sections":["zkufh760uvF3"]},"kernelspec":{"display_name":"Python 3","name":"python3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"zkufh760uvF3"},"source":["![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n","\n","[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/Training/multi_lingual/multi_class_text_classification/NLU_training_multi_lingual_multi_class_text_classifier_demo_amazon.ipynb)\n","\n","\n","\n","\n","# Training a Deep Learning Classifier with NLU \n","## ClassifierDL (Multi-class Text Classification)\n","## 3 class Amazon Phone review classifier training]\n","With the [ClassifierDL model](https://nlp.johnsnowlabs.com/docs/en/annotators#classifierdl-multi-class-text-classification) from Spark NLP you can achieve State Of the Art results on any multi class text classification problem \n","\n","This notebook showcases the following features : \n","\n","- How to train the deep learning classifier\n","- How to store a pipeline to disk\n","- How to load the pipeline from disk (Enables NLU offline mode)\n","\n","\n","You can achieve these results or even better on this dataset with training data :\n","\n","<br>\n","\n","\n","![image.png]()\n","\n","\n","\n","You can achieve these results or even better on this dataset with test data :\n","\n","<br>\n","\n","![image.png]()"]},{"cell_type":"markdown","metadata":{"id":"dur2drhW5Rvi"},"source":["# 1. Install Java 8 and NLU"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"hFGnBCHavltY","executionInfo":{"status":"ok","timestamp":1620206587860,"user_tz":-300,"elapsed":34820,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"f6325653-98eb-43e7-8621-b624faaca59b"},"source":["!wget https://setup.johnsnowlabs.com/nlu/colab.sh -O - | bash\n","  \n","\n","import nlu"],"execution_count":null,"outputs":[{"output_type":"stream","text":["--2021-05-05 09:22:33--  https://raw.githubusercontent.com/JohnSnowLabs/nlu/master/scripts/colab_setup.sh\n","Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.111.133, ...\n","Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 1671 (1.6K) [text/plain]\n","Saving to: ‘STDOUT’\n","\n","-                   100%[===================>]   1.63K  --.-KB/s    in 0s      \n","\n","2021-05-05 09:22:33 (28.5 MB/s) - written to stdout [1671/1671]\n","\n","Installing  NLU 3.0.0 with  PySpark 3.0.2 and Spark NLP 3.0.1 for Google Colab ...\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"f4KkTfnR5Ugg"},"source":["# 2. Download Amazon Unlocked mobile phones dataset \n","https://www.kaggle.com/PromptCloudHQ/amazon-reviews-unlocked-mobile-phones\n","\n","dataset with unlocked mobile phone reviews in 5 review classes\n"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"OrVb5ZMvvrQD","executionInfo":{"status":"ok","timestamp":1620206589024,"user_tz":-300,"elapsed":35970,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"094fa652-e925-4360-80c2-fd54b806284e"},"source":["! wget http://ckl-it.de/wp-content/uploads/2021/02/Amazon_Unlocked_Mobile_multi_lingual.csv"],"execution_count":null,"outputs":[{"output_type":"stream","text":["--2021-05-05 09:23:07--  http://ckl-it.de/wp-content/uploads/2021/02/Amazon_Unlocked_Mobile_multi_lingual.csv\n","Resolving ckl-it.de (ckl-it.de)... 217.160.0.108, 2001:8d8:100f:f000::209\n","Connecting to ckl-it.de (ckl-it.de)|217.160.0.108|:80... connected.\n","HTTP request sent, awaiting response... 200 OK\n","Length: 511871 (500K) [text/csv]\n","Saving to: ‘Amazon_Unlocked_Mobile_multi_lingual.csv.1’\n","\n","Amazon_Unlocked_Mob 100%[===================>] 499.87K   810KB/s    in 0.6s    \n","\n","2021-05-05 09:23:08 (810 KB/s) - ‘Amazon_Unlocked_Mobile_multi_lingual.csv.1’ saved [511871/511871]\n","\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":419},"id":"y4xSRWIhwT28","executionInfo":{"status":"ok","timestamp":1620206589583,"user_tz":-300,"elapsed":36515,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"e7079afb-8f0a-41d4-8a77-560b175ca7e6"},"source":["import pandas as pd\n","test_path = '/content/Amazon_Unlocked_Mobile_multi_lingual.csv'\n","train_df = pd.read_csv(test_path,sep=\",\")\n","cols = [\"y\",\"text\"]\n","train_df = train_df[cols]\n","from sklearn.model_selection import train_test_split\n","train_df, test_df = train_test_split(train_df, test_size=0.2)\n","train_df\n","\n"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>y</th>\n","      <th>text</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>1266</th>\n","      <td>good</td>\n","      <td>This is like the 5th cellphone for my dad in l...</td>\n","    </tr>\n","    <tr>\n","      <th>121</th>\n","      <td>poor</td>\n","      <td>I bought this phone to give as a gift to a fri...</td>\n","    </tr>\n","    <tr>\n","      <th>528</th>\n","      <td>good</td>\n","      <td>nice phone, nice up grade from my pantach revu...</td>\n","    </tr>\n","    <tr>\n","      <th>151</th>\n","      <td>good</td>\n","      <td>Ver Good!</td>\n","    </tr>\n","    <tr>\n","      <th>892</th>\n","      <td>good</td>\n","      <td>excellent product in perfect condition</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>1490</th>\n","      <td>average</td>\n","      <td>So far so good. Needed a stopgap for my old Ne...</td>\n","    </tr>\n","    <tr>\n","      <th>894</th>\n","      <td>average</td>\n","      <td>Good phone overall. Excellent speakers and ver...</td>\n","    </tr>\n","    <tr>\n","      <th>314</th>\n","      <td>good</td>\n","      <td>I love it, the only problem is the camera its ...</td>\n","    </tr>\n","    <tr>\n","      <th>53</th>\n","      <td>average</td>\n","      <td>The battery goes down to quick, takes a while ...</td>\n","    </tr>\n","    <tr>\n","      <th>739</th>\n","      <td>average</td>\n","      <td>I\"m giving this phone a 3 because there seems ...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>1200 rows × 2 columns</p>\n","</div>"],"text/plain":["            y                                               text\n","1266     good  This is like the 5th cellphone for my dad in l...\n","121      poor  I bought this phone to give as a gift to a fri...\n","528      good  nice phone, nice up grade from my pantach revu...\n","151      good                                          Ver Good!\n","892      good             excellent product in perfect condition\n","...       ...                                                ...\n","1490  average  So far so good. Needed a stopgap for my old Ne...\n","894   average  Good phone overall. Excellent speakers and ver...\n","314      good  I love it, the only problem is the camera its ...\n","53    average  The battery goes down to quick, takes a while ...\n","739   average  I\"m giving this phone a 3 because there seems ...\n","\n","[1200 rows x 2 columns]"]},"metadata":{"tags":[]},"execution_count":5}]},{"cell_type":"markdown","metadata":{"id":"0296Om2C5anY"},"source":["# 3. Train Deep Learning Classifier using nlu.load('train.classifier')\n","\n","You dataset label column should be named 'y' and the feature column with text data should be named 'text'"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/","height":861},"id":"3ZIPkRkWftBG","executionInfo":{"status":"ok","timestamp":1620208183255,"user_tz":-300,"elapsed":41782,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"1d50fa2f-bd42-41f8-dd24-5e170ec8366c"},"source":["trainable_pipe = nlu.load('xx.embed_sentence.labse train.classifier')\n","# We need to train longer and user smaller LR for NON-USE based sentence embeddings usually\n","# We could tune the hyperparameters further with hyperparameter tuning methods like gridsearch\n","# Also longer training gives more accuracy\n","trainable_pipe['trainable_classifier_dl'].setMaxEpochs(60)  \n","trainable_pipe['trainable_classifier_dl'].setLr(0.005) \n","fitted_pipe = trainable_pipe.fit(train_df)\n","# predict with the trainable pipeline on dataset and get predictions\n","preds = fitted_pipe.predict(train_df,output_level='document')\n","\n","#sentence detector that is part of the pipe generates sone NaNs. lets drop them first\n","preds.dropna(inplace=True)\n","from sklearn.metrics import classification_report\n","print(classification_report(preds['y'], preds['classifier_dl']))\n","\n","preds"],"execution_count":null,"outputs":[{"output_type":"stream","text":["labse download started this may take some time.\n","Approximate size to download 1.7 GB\n","[OK!]\n","sentence_detector_dl download started this may take some time.\n","Approximate size to download 354.6 KB\n","[OK!]\n","              precision    recall  f1-score   support\n","\n","     average       0.78      0.84      0.81       390\n","        good       0.87      0.91      0.89       414\n","        poor       0.92      0.81      0.86       396\n","\n","    accuracy                           0.85      1200\n","   macro avg       0.86      0.85      0.85      1200\n","weighted avg       0.86      0.85      0.85      1200\n","\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>text</th>\n","      <th>sentence</th>\n","      <th>y</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>This is like the 5th cellphone for my dad in l...</td>\n","      <td>good</td>\n","      <td>1266</td>\n","      <td>0.990780</td>\n","      <td>[-0.04503230005502701, -0.0068597435019910336,...</td>\n","      <td>This is like the 5th cellphone for my dad in l...</td>\n","      <td>[This is like the 5th cellphone for my dad in ...</td>\n","      <td>good</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>I bought this phone to give as a gift to a fri...</td>\n","      <td>poor</td>\n","      <td>121</td>\n","      <td>0.995079</td>\n","      <td>[-0.053808409720659256, 0.043268490582704544, ...</td>\n","      <td>I bought this phone to give as a gift to a fri...</td>\n","      <td>[I bought this phone to give as a gift to a fr...</td>\n","      <td>poor</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>nice phone, nice up grade from my pantach revu...</td>\n","      <td>good</td>\n","      <td>528</td>\n","      <td>1.000000</td>\n","      <td>[-0.031718023121356964, 0.05311667174100876, -...</td>\n","      <td>nice phone, nice up grade from my pantach revu...</td>\n","      <td>[nice phone, nice up grade from my pantach rev...</td>\n","      <td>good</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>Ver Good!</td>\n","      <td>good</td>\n","      <td>151</td>\n","      <td>0.999916</td>\n","      <td>[0.004338219296187162, -0.05601995438337326, -...</td>\n","      <td>Ver Good!</td>\n","      <td>[Ver Good!]</td>\n","      <td>good</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>excellent product in perfect condition</td>\n","      <td>good</td>\n","      <td>892</td>\n","      <td>0.999986</td>\n","      <td>[-0.045638032257556915, 0.013802768662571907, ...</td>\n","      <td>excellent product in perfect condition</td>\n","      <td>[excellent product in perfect condition]</td>\n","      <td>good</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>1195</th>\n","      <td>So far so good. Needed a stopgap for my old Ne...</td>\n","      <td>average</td>\n","      <td>1490</td>\n","      <td>0.997774</td>\n","      <td>[-0.03902469947934151, 0.03762187063694, -0.01...</td>\n","      <td>So far so good. Needed a stopgap for my old Ne...</td>\n","      <td>[So far so good., Needed a stopgap for my old ...</td>\n","      <td>average</td>\n","    </tr>\n","    <tr>\n","      <th>1196</th>\n","      <td>Good phone overall. Excellent speakers and ver...</td>\n","      <td>average</td>\n","      <td>894</td>\n","      <td>0.999868</td>\n","      <td>[-0.0373789444565773, -0.011103338561952114, -...</td>\n","      <td>Good phone overall. Excellent speakers and ver...</td>\n","      <td>[Good phone overall., Excellent speakers and v...</td>\n","      <td>average</td>\n","    </tr>\n","    <tr>\n","      <th>1197</th>\n","      <td>I love it, the only problem is the camera its ...</td>\n","      <td>average</td>\n","      <td>314</td>\n","      <td>0.999975</td>\n","      <td>[0.0009155190782621503, -0.04911276325583458, ...</td>\n","      <td>I love it, the only problem is the camera its ...</td>\n","      <td>[I love it, the only problem is the camera its...</td>\n","      <td>good</td>\n","    </tr>\n","    <tr>\n","      <th>1198</th>\n","      <td>The battery goes down to quick, takes a while ...</td>\n","      <td>poor</td>\n","      <td>53</td>\n","      <td>0.535710</td>\n","      <td>[-0.06607282906770706, 0.012522447854280472, -...</td>\n","      <td>The battery goes down to quick, takes a while ...</td>\n","      <td>[The battery goes down to quick, takes a while...</td>\n","      <td>average</td>\n","    </tr>\n","    <tr>\n","      <th>1199</th>\n","      <td>I\"m giving this phone a 3 because there seems ...</td>\n","      <td>average</td>\n","      <td>739</td>\n","      <td>1.000000</td>\n","      <td>[-0.04678669199347496, -0.02481876127421856, -...</td>\n","      <td>I\"m giving this phone a 3 because there seems ...</td>\n","      <td>[I\"m giving this phone a 3 because there seems...</td>\n","      <td>average</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>1200 rows × 8 columns</p>\n","</div>"],"text/plain":["                                               document  ...        y\n","0     This is like the 5th cellphone for my dad in l...  ...     good\n","1     I bought this phone to give as a gift to a fri...  ...     poor\n","2     nice phone, nice up grade from my pantach revu...  ...     good\n","3                                             Ver Good!  ...     good\n","4                excellent product in perfect condition  ...     good\n","...                                                 ...  ...      ...\n","1195  So far so good. Needed a stopgap for my old Ne...  ...  average\n","1196  Good phone overall. Excellent speakers and ver...  ...  average\n","1197  I love it, the only problem is the camera its ...  ...     good\n","1198  The battery goes down to quick, takes a while ...  ...  average\n","1199  I\"m giving this phone a 3 because there seems ...  ...  average\n","\n","[1200 rows x 8 columns]"]},"metadata":{"tags":[]},"execution_count":6}]},{"cell_type":"markdown","metadata":{"id":"_1jxw3GnVGlI"},"source":["# 3.1 evaluate on Test Data"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"Fxx4yNkNVGFl","executionInfo":{"status":"ok","timestamp":1620208254188,"user_tz":-300,"elapsed":70962,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"5188c09f-0fdc-417a-fb64-0edfb8b57b37"},"source":["preds = fitted_pipe.predict(test_df,output_level='document')\n","\n","#sentence detector that is part of the pipe generates sone NaNs. lets drop them first\n","preds.dropna(inplace=True)\n","print(classification_report(preds['y'], preds['classifier_dl']))"],"execution_count":null,"outputs":[{"output_type":"stream","text":["              precision    recall  f1-score   support\n","\n","     average       0.68      0.75      0.71       110\n","        good       0.76      0.80      0.78        86\n","        poor       0.79      0.67      0.73       104\n","\n","    accuracy                           0.74       300\n","   macro avg       0.74      0.74      0.74       300\n","weighted avg       0.74      0.74      0.74       300\n","\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"yUkk_L8MGcRg"},"source":["#4. Test Model  with  20 languages!"]},{"cell_type":"code","metadata":{"id":"q2s6nsZZGcRm","colab":{"base_uri":"https://localhost:8080/","height":759},"executionInfo":{"status":"ok","timestamp":1620208282713,"user_tz":-300,"elapsed":28530,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"e797e94f-d83c-494e-ba0d-d7887543b42b"},"source":["train_df = pd.read_csv(\"Amazon_Unlocked_Mobile_multi_lingual.csv\")\n","preds = fitted_pipe.predict(train_df[[\"test_sentences\",\"y\"]].iloc[:100],output_level='document')\n","\n","#sentence detector that is part of the pipe generates sone NaNs. lets drop them first\n","preds.dropna(inplace=True)\n","print(classification_report(preds['y'], preds['classifier_dl']))\n","\n","preds"],"execution_count":null,"outputs":[{"output_type":"stream","text":["              precision    recall  f1-score   support\n","\n","     average       0.65      0.90      0.75        29\n","        good       0.85      0.88      0.86        32\n","        poor       1.00      0.69      0.82        39\n","\n","    accuracy                           0.81       100\n","   macro avg       0.83      0.82      0.81       100\n","weighted avg       0.85      0.81      0.81       100\n","\n"],"name":"stdout"},{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>text</th>\n","      <th>sentence</th>\n","      <th>y</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Alındı, onu yandırdı, işləmədi. Geri açıldı, b...</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999992</td>\n","      <td>[0.023933352902531624, 0.03197602182626724, -0...</td>\n","      <td>Alındı, onu yandırdı, işləmədi. Geri açıldı, b...</td>\n","      <td>[Alındı, onu yandırdı, işləmədi., Geri açıldı,...</td>\n","      <td>poor</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>דערווייַל עס איז 2014, די 3gs איז דיסקאַנטיניו...</td>\n","      <td>average</td>\n","      <td>1</td>\n","      <td>1.000000</td>\n","      <td>[-0.019562795758247375, -0.03646012768149376, ...</td>\n","      <td>דערווייַל עס איז 2014, די 3gs איז דיסקאַנטיניו...</td>\n","      <td>[דערווייַל עס איז 2014, די 3gs איז דיסקאַנטיני...</td>\n","      <td>average</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>100% recommended</td>\n","      <td>good</td>\n","      <td>2</td>\n","      <td>0.999967</td>\n","      <td>[0.022297078743577003, -0.038920555263757706, ...</td>\n","      <td>100% recommended</td>\n","      <td>[100% recommended]</td>\n","      <td>good</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>Đó là một chiếc điện thoại tốt nhưng nếu bạn s...</td>\n","      <td>good</td>\n","      <td>3</td>\n","      <td>0.999701</td>\n","      <td>[0.023473074659705162, -0.056649111211299896, ...</td>\n","      <td>Đó là một chiếc điện thoại tốt nhưng nếu bạn s...</td>\n","      <td>[Đó là một chiếc điện thoại tốt nhưng nếu bạn ...</td>\n","      <td>average</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>វាល្អដែលទូរស័ព្ទនេះមានប្រព័ន្ធ LTE ហើយវាដំណើរក...</td>\n","      <td>average</td>\n","      <td>4</td>\n","      <td>1.000000</td>\n","      <td>[-0.04908803477883339, 0.006140733137726784, -...</td>\n","      <td>វាល្អដែលទូរស័ព្ទនេះមានប្រព័ន្ធ LTE ហើយវាដំណើរក...</td>\n","      <td>[វាល្អដែលទូរស័ព្ទនេះមានប្រព័ន្ធ LTE ហើយវាដំណើរ...</td>\n","      <td>average</td>\n","    </tr>\n","    <tr>\n","      <th>...</th>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","      <td>...</td>\n","    </tr>\n","    <tr>\n","      <th>95</th>\n","      <td>העלא, מיר געקויפט צוויי פאָנעס און זיי געקומען...</td>\n","      <td>average</td>\n","      <td>95</td>\n","      <td>0.994167</td>\n","      <td>[-0.05593854561448097, 0.04983929917216301, -0...</td>\n","      <td>העלא, מיר געקויפט צוויי פאָנעס און זיי געקומען...</td>\n","      <td>[העלא, מיר געקויפט צוויי פאָנעס און זיי געקומע...</td>\n","      <td>poor</td>\n","    </tr>\n","    <tr>\n","      <th>96</th>\n","      <td>Uitstekend</td>\n","      <td>good</td>\n","      <td>96</td>\n","      <td>1.000000</td>\n","      <td>[0.017209608107805252, 0.013231031596660614, -...</td>\n","      <td>Uitstekend</td>\n","      <td>[Uitstekend]</td>\n","      <td>good</td>\n","    </tr>\n","    <tr>\n","      <th>97</th>\n","      <td>پروڈکٹ اچھی ہے لیکن انگریزی زبان میں اب بھی چی...</td>\n","      <td>average</td>\n","      <td>97</td>\n","      <td>0.999992</td>\n","      <td>[-0.0447855070233345, 0.03711877763271332, -0....</td>\n","      <td>پروڈکٹ اچھی ہے لیکن انگریزی زبان میں اب بھی چی...</td>\n","      <td>[پروڈکٹ اچھی ہے لیکن انگریزی زبان میں اب بھی چ...</td>\n","      <td>poor</td>\n","    </tr>\n","    <tr>\n","      <th>98</th>\n","      <td>Veronderstel om 'n splinternuwe ontsluitfoon t...</td>\n","      <td>poor</td>\n","      <td>98</td>\n","      <td>0.999984</td>\n","      <td>[-0.0475904643535614, 0.04630507901310921, -0....</td>\n","      <td>Veronderstel om 'n splinternuwe ontsluitfoon t...</td>\n","      <td>[Veronderstel om 'n splinternuwe ontsluitfoon ...</td>\n","      <td>poor</td>\n","    </tr>\n","    <tr>\n","      <th>99</th>\n","      <td>خلل بسيط ومزعج للغاية عند إرسال الرسائل النصية...</td>\n","      <td>average</td>\n","      <td>99</td>\n","      <td>0.943725</td>\n","      <td>[-0.027637170627713203, 0.0048340680077672005,...</td>\n","      <td>خلل بسيط ومزعج للغاية عند إرسال الرسائل النصية...</td>\n","      <td>[خلل بسيط ومزعج للغاية عند إرسال الرسائل النصي...</td>\n","      <td>average</td>\n","    </tr>\n","  </tbody>\n","</table>\n","<p>100 rows × 8 columns</p>\n","</div>"],"text/plain":["                                             document  ...        y\n","0   Alındı, onu yandırdı, işləmədi. Geri açıldı, b...  ...     poor\n","1   דערווייַל עס איז 2014, די 3gs איז דיסקאַנטיניו...  ...  average\n","2                                    100% recommended  ...     good\n","3   Đó là một chiếc điện thoại tốt nhưng nếu bạn s...  ...  average\n","4   វាល្អដែលទូរស័ព្ទនេះមានប្រព័ន្ធ LTE ហើយវាដំណើរក...  ...  average\n","..                                                ...  ...      ...\n","95  העלא, מיר געקויפט צוויי פאָנעס און זיי געקומען...  ...     poor\n","96                                         Uitstekend  ...     good\n","97  پروڈکٹ اچھی ہے لیکن انگریزی زبان میں اب بھی چی...  ...     poor\n","98  Veronderstel om 'n splinternuwe ontsluitfoon t...  ...     poor\n","99  خلل بسيط ومزعج للغاية عند إرسال الرسائل النصية...  ...  average\n","\n","[100 rows x 8 columns]"]},"metadata":{"tags":[]},"execution_count":8}]},{"cell_type":"markdown","metadata":{"id":"RjtuNUcvuJTT"},"source":["# The Model understands Englsih\n","![en](https://www.worldometers.info/img/flags/small/tn_nz-flag.gif)"]},{"cell_type":"code","metadata":{"id":"o0vu7PaWkcI7","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208284226,"user_tz":-300,"elapsed":1537,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"f2326b4b-a210-4515-d10e-cf965b17af12"},"source":["fitted_pipe.predict(\"It was like brand new \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>It was like brand new</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.971137</td>\n","      <td>[0.02449253760278225, -0.003671379294246435, -...</td>\n","      <td>[It was like brand new]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                document  ...                 sentence\n","0  It was like brand new  ...  [It was like brand new]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":9}]},{"cell_type":"code","metadata":{"id":"1ykjRQhCtQ4w","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208285058,"user_tz":-300,"elapsed":843,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"99fc430e-7de4-44e5-d002-5b1f60d8137c"},"source":["fitted_pipe.predict(\"It stopped working on the first day \")\n"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>It stopped working on the first day</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999895</td>\n","      <td>[-0.0048237149603664875, 0.020508447661995888,...</td>\n","      <td>[It stopped working on the first day]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                              document  ...                               sentence\n","0  It stopped working on the first day  ...  [It stopped working on the first day]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":10}]},{"cell_type":"markdown","metadata":{"id":"vohym-XbuNHn"},"source":["# The Model understands German\n","![de](https://www.worldometers.info/img/flags/small/tn_gm-flag.gif)"]},{"cell_type":"code","metadata":{"id":"dzaaZrI4tVWc","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208286338,"user_tz":-300,"elapsed":1290,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"a122f086-37e9-4273-9f36-d5742d05b42e"},"source":["# German for: 'It worked perfectly '\n","fitted_pipe.predict(\"Es hat perfekt funktioniert\")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Es hat perfekt funktioniert</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.998511</td>\n","      <td>[-0.005111832171678543, -0.048203449696302414,...</td>\n","      <td>[Es hat perfekt funktioniert]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                      document  ...                       sentence\n","0  Es hat perfekt funktioniert  ...  [Es hat perfekt funktioniert]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":11}]},{"cell_type":"code","metadata":{"id":"BbhgTSBGtTtJ","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208287189,"user_tz":-300,"elapsed":862,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"2454f205-eb8c-488a-9883-d555e4ab9849"},"source":["# German for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"Am ersten Tag hörte es auf zu arbeiten \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Am ersten Tag hörte es auf zu arbeiten</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999722</td>\n","      <td>[0.02086009830236435, -0.011390610598027706, 0...</td>\n","      <td>[Am ersten Tag hörte es auf zu arbeiten]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                 document  ...                                  sentence\n","0  Am ersten Tag hörte es auf zu arbeiten  ...  [Am ersten Tag hörte es auf zu arbeiten]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":12}]},{"cell_type":"markdown","metadata":{"id":"a1JbtmWquQwj"},"source":["# The Model understands Chinese\n","![zh](https://www.worldometers.info/img/flags/small/tn_ch-flag.gif)"]},{"cell_type":"code","metadata":{"id":"kYSYqtoRtc-P","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208288045,"user_tz":-300,"elapsed":867,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"df22eb72-fe4f-4c33-e42b-355393886cb7"},"source":["# Chinese for: 'It was like brand new'\n","fitted_pipe.predict(\"就像全新 \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>就像全新</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.999759</td>\n","      <td>[-0.018629543483257294, -0.023574186488986015,...</td>\n","      <td>[就像全新]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["  document  ... sentence\n","0     就像全新  ...   [就像全新]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":13}]},{"cell_type":"code","metadata":{"id":"06v9SD-QtlBU","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208288891,"user_tz":-300,"elapsed":853,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"40291b56-9ab4-4283-a895-f561cda66d88"},"source":["# Chinese for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"第一天停止工作 \")\n","\t\t"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>第一天停止工作</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.99838</td>\n","      <td>[-0.0022839070297777653, 0.01226264052093029, ...</td>\n","      <td>[第一天停止工作]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["  document  ...   sentence\n","0  第一天停止工作  ...  [第一天停止工作]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":14}]},{"cell_type":"markdown","metadata":{"id":"9h7CvN4uu9Pb"},"source":["# Model understands Afrikaans\n","\n","![af](https://www.worldometers.info/img/flags/small/tn_sf-flag.gif)\n","\n"]},{"cell_type":"code","metadata":{"id":"VMPhbgw9twtf","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208290261,"user_tz":-300,"elapsed":1378,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"0c85cc35-f3f0-4406-be49-220f754de617"},"source":["\n","#  Afrikaans for: 'It worked perfectly '\n","fitted_pipe.predict(\"Dit het perfek gewerk\")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Dit het perfek gewerk</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.997483</td>\n","      <td>[-0.001879673101939261, -0.043611448258161545,...</td>\n","      <td>[Dit het perfek gewerk]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                document  ...                 sentence\n","0  Dit het perfek gewerk  ...  [Dit het perfek gewerk]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":15}]},{"cell_type":"code","metadata":{"id":"zWgNTIdkumhX","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208290606,"user_tz":-300,"elapsed":353,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"89c5e830-650e-47a6-a2a6-f31ccea2a77f"},"source":["#  Afrikaans for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"Dit het op die eerste dag opgehou werk \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Dit het op die eerste dag opgehou werk</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999799</td>\n","      <td>[0.00801782961934805, -0.01342733483761549, -0...</td>\n","      <td>[Dit het op die eerste dag opgehou werk]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                 document  ...                                  sentence\n","0  Dit het op die eerste dag opgehou werk  ...  [Dit het op die eerste dag opgehou werk]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":16}]},{"cell_type":"markdown","metadata":{"id":"GITfT7FK0CGv"},"source":["# The model understands Zulu\n","![zu](https://www.worldometers.info/img/flags/small/tn_sf-flag.gif)"]},{"cell_type":"code","metadata":{"id":"ifRhs6e7OcR3","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208291786,"user_tz":-300,"elapsed":1194,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"2282c562-dd97-4784-f2ee-8114ad955dae"},"source":["#  Zulu for: 'It worked perfectly '\n","fitted_pipe.predict(\"Kusebenze ngokuphelele\")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Kusebenze ngokuphelele</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.675412</td>\n","      <td>[0.010045904666185379, -0.05170843377709389, -...</td>\n","      <td>[Kusebenze ngokuphelele]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                 document  ...                  sentence\n","0  Kusebenze ngokuphelele  ...  [Kusebenze ngokuphelele]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":17}]},{"cell_type":"code","metadata":{"id":"6uelDwq4xdWv","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208292579,"user_tz":-300,"elapsed":801,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"4ed1017b-42b9-4635-eb61-52aa29987da0"},"source":["#  Zulu for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"Iyeke ukusebenza ngosuku lokuqala \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Iyeke ukusebenza ngosuku lokuqala</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.9992</td>\n","      <td>[0.004491243977099657, 0.018703386187553406, 0...</td>\n","      <td>[Iyeke ukusebenza ngosuku lokuqala]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                            document  ...                             sentence\n","0  Iyeke ukusebenza ngosuku lokuqala  ...  [Iyeke ukusebenza ngosuku lokuqala]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":18}]},{"cell_type":"markdown","metadata":{"id":"VGVvzl_30a0T"},"source":["# The  Model understands Turkish\n","![tr](https://www.worldometers.info/img/flags/small/tn_tu-flag.gif)"]},{"cell_type":"code","metadata":{"id":"DRNnuEeQz2pd","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208293097,"user_tz":-300,"elapsed":528,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"0d9ba936-54c4-41d8-a9b2-4f4cd18a6b7d"},"source":["\n","#  Turkish for: 'It It worked perfectly '\n","fitted_pipe.predict(\"Mükemmel çalıştı\")\n","\t\t"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Mükemmel çalıştı</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.999996</td>\n","      <td>[0.06036874279379845, 0.0009111528052017093, -...</td>\n","      <td>[Mükemmel çalıştı]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["           document  ...            sentence\n","0  Mükemmel çalıştı  ...  [Mükemmel çalıştı]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":19}]},{"cell_type":"code","metadata":{"id":"aOSsiK6J0jWs","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208294444,"user_tz":-300,"elapsed":790,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"20619d0e-c9ba-4862-c953-7247b747d87a"},"source":["\n","#  Turkish for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"İlk gün çalışmayı bıraktı \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>İlk gün çalışmayı bıraktı</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.998341</td>\n","      <td>[0.050941046327352524, 0.020712943747639656, 0...</td>\n","      <td>[İlk gün çalışmayı bıraktı]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                    document  ...                     sentence\n","0  İlk gün çalışmayı bıraktı  ...  [İlk gün çalışmayı bıraktı]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":20}]},{"cell_type":"markdown","metadata":{"id":"803qL2gt0vlb"},"source":["#  The Model understands Hebrew\n","![he](https://www.worldometers.info/img/flags/small/tn_sf-flag.gif)"]},{"cell_type":"code","metadata":{"id":"XQ5VCtxw0pc0","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208295262,"user_tz":-300,"elapsed":830,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"c12a5cd4-2c49-4807-bfcb-56232c7c7c90"},"source":["# Hebrew for: 'It worked perfectly '\n","fitted_pipe.predict(\"זה עבד בצורה מושלמת\")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>זה עבד בצורה מושלמת</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.888842</td>\n","      <td>[-0.01338683720678091, -0.054987359791994095, ...</td>\n","      <td>[זה עבד בצורה מושלמת]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["              document  ...               sentence\n","0  זה עבד בצורה מושלמת  ...  [זה עבד בצורה מושלמת]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":21}]},{"cell_type":"code","metadata":{"id":"9w2ZHfns05A4","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208296285,"user_tz":-300,"elapsed":1030,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"229daf7b-d6bc-4819-c228-fb75068e6239"},"source":["\t\t\n","# Hebrew for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"זה הפסיק לעבוד ביום הראשון \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>זה הפסיק לעבוד ביום הראשון</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999894</td>\n","      <td>[-0.013081019744277, -0.02689044550061226, -0....</td>\n","      <td>[זה הפסיק לעבוד ביום הראשון]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                     document  ...                      sentence\n","0  זה הפסיק לעבוד ביום הראשון  ...  [זה הפסיק לעבוד ביום הראשון]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":22}]},{"cell_type":"markdown","metadata":{"id":"SDlpd33H1HIX"},"source":["# The Model understands Telugu\n","![te](https://www.worldometers.info/img/flags/small/tn_in-flag.gif)\n"]},{"cell_type":"code","metadata":{"id":"Kc5n1bzv1BJT","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208297143,"user_tz":-300,"elapsed":865,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"6e51fe15-e745-4c7a-f996-f38b04f993e7"},"source":["\t\t\n","# Telugu for: 'It was like brand new'\n","fitted_pipe.predict(\"ఇది సరికొత్తది \")\n","\t\t"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>ఇది సరికొత్తది</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.99935</td>\n","      <td>[0.020253609865903854, -0.045859843492507935, ...</td>\n","      <td>[ఇది సరికొత్తది]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["         document  ...          sentence\n","0  ఇది సరికొత్తది  ...  [ఇది సరికొత్తది]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":23}]},{"cell_type":"code","metadata":{"id":"-l-u6vrz1Obe","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208297504,"user_tz":-300,"elapsed":374,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"1119d43e-fb8a-4588-b515-6e57690ba3cb"},"source":["\n","# Telugu for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"ఇది మొదటి రోజు పనిచేయడం మానేసింది \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>ఇది మొదటి రోజు పనిచేయడం మానేసింది</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999886</td>\n","      <td>[0.00022219969832804054, -0.01876474916934967,...</td>\n","      <td>[ఇది మొదటి రోజు పనిచేయడం మానేసింది]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                            document  ...                             sentence\n","0  ఇది మొదటి రోజు పనిచేయడం మానేసింది  ...  [ఇది మొదటి రోజు పనిచేయడం మానేసింది]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":24}]},{"cell_type":"markdown","metadata":{"id":"nziBUe8t1Zwn"},"source":["# Model understands Russian\n","![ru](https://www.worldometers.info/img/flags/small/tn_rs-flag.gif)\n"]},{"cell_type":"code","metadata":{"id":"Ckyjl3YQ1VFn","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208298684,"user_tz":-300,"elapsed":1191,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"1c3b0bde-17c4-483b-9ee6-54f20af2ee9f"},"source":["\t\t\n","#  Russian for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"Перестал работать в первый же день \")\n","\t\t"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Перестал работать в первый же день</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999956</td>\n","      <td>[-0.040550969541072845, 0.023711256682872772, ...</td>\n","      <td>[Перестал работать в первый же день]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                             document  ...                              sentence\n","0  Перестал работать в первый же день  ...  [Перестал работать в первый же день]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":25}]},{"cell_type":"code","metadata":{"id":"GIdWkfGv1gFz","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208299374,"user_tz":-300,"elapsed":706,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"5276251f-bd2d-4503-9af9-bddf41340558"},"source":["#  Russian for: 'It worked perfectly '\n","fitted_pipe.predict(\"Это сработало отлично\")\n"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Это сработало отлично</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.992138</td>\n","      <td>[0.008218108676373959, -0.05058329924941063, -...</td>\n","      <td>[Это сработало отлично]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                document  ...                 sentence\n","0  Это сработало отлично  ...  [Это сработало отлично]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":26}]},{"cell_type":"markdown","metadata":{"id":"HKj5yWwwMplH"},"source":["# The Model understands French\n","![fr](https://www.worldometers.info/img/flags/small/tn_fr-flag.gif)"]},{"cell_type":"code","metadata":{"id":"CUHcJZfJMplL","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208304798,"user_tz":-300,"elapsed":562,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"e3b7057e-3ee0-4c1f-9a71-58f2caab31bf"},"source":["# French for: 'It was the best ever !!'\n","fitted_pipe.predict(\"C'était la meilleure chose que je n'ai jamais faite !!\")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>C'était la meilleure chose que je n'ai jamais ...</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.789434</td>\n","      <td>[0.027519920840859413, -0.04782726615667343, -...</td>\n","      <td>[C'était la meilleure chose que je n'ai jamais...</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                            document  ...                                           sentence\n","0  C'était la meilleure chose que je n'ai jamais ...  ...  [C'était la meilleure chose que je n'ai jamais...\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":33}]},{"cell_type":"code","metadata":{"id":"57NY2XoTMplM","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208305800,"user_tz":-300,"elapsed":1014,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"5865f7da-7bfc-4f4e-ca1e-8366f10441bf"},"source":["\t\t\n","# French for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"Il a cessé de fonctionner le premier jour \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Il a cessé de fonctionner le premier jour</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.99996</td>\n","      <td>[-0.02747691236436367, -0.006572246551513672, ...</td>\n","      <td>[Il a cessé de fonctionner le premier jour]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                    document  ...                                     sentence\n","0  Il a cessé de fonctionner le premier jour  ...  [Il a cessé de fonctionner le premier jour]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":34}]},{"cell_type":"markdown","metadata":{"id":"jD2TBgT0Nq6F"},"source":["# The Model understands Thai\n","![th](https://www.worldometers.info/img/flags/small/tn_th-flag.gif)"]},{"cell_type":"code","metadata":{"id":"gBp11S5GNq6S","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208306839,"user_tz":-300,"elapsed":1046,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"2f8d6182-9714-4b8e-8290-7751654f680f"},"source":["# Thai for: 'It was the best ever !!'\n","fitted_pipe.predict(\"มันดีที่สุดเท่าที่เคยมีมา !!\")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>มันดีที่สุดเท่าที่เคยมีมา !!</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.9783</td>\n","      <td>[-0.00953330472111702, -0.05253228917717934, -...</td>\n","      <td>[มันดีที่สุดเท่าที่เคยมีมา !!]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                       document  ...                        sentence\n","0  มันดีที่สุดเท่าที่เคยมีมา !!  ...  [มันดีที่สุดเท่าที่เคยมีมา !!]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":35}]},{"cell_type":"code","metadata":{"id":"R6nKI7C3QKa3","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208307547,"user_tz":-300,"elapsed":714,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"89d73606-66a5-4a1c-ea7d-f5dfcf93d8be"},"source":["# Thai for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"มันหยุดทำงานในวันแรก \")\n","\t\t"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>มันหยุดทำงานในวันแรก</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999704</td>\n","      <td>[-0.02828541025519371, -0.025081545114517212, ...</td>\n","      <td>[มันหยุดทำงานในวันแรก]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["               document  ...                sentence\n","0  มันหยุดทำงานในวันแรก  ...  [มันหยุดทำงานในวันแรก]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":36}]},{"cell_type":"markdown","metadata":{"id":"mLItI4KZOElB"},"source":["# The Model understands Khmer\n","![km](https://www.worldometers.info/img/flags/small/tn_cb-flag.gif)"]},{"cell_type":"code","metadata":{"id":"SWbqMgAwOElC","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208307858,"user_tz":-300,"elapsed":357,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"2b9cb7bf-7280-4e50-946f-7d27d735249b"},"source":["\n","# Khmer for: 'It was like brand new'\n","fitted_pipe.predict(\"វាដូចជាម៉ាកថ្មី \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>វាដូចជាម៉ាកថ្មី</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.998938</td>\n","      <td>[-0.013914491981267929, 0.01159849762916565, -...</td>\n","      <td>[វាដូចជាម៉ាកថ្មី]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["          document  ...           sentence\n","0  វាដូចជាម៉ាកថ្មី  ...  [វាដូចជាម៉ាកថ្មី]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":37}]},{"cell_type":"code","metadata":{"id":"beoCtm4xQf2P","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208308862,"user_tz":-300,"elapsed":1021,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"c2c9f684-ef2d-4bb3-bc16-7afdda19fe30"},"source":["\t\t\n","# Khmer for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"វាឈប់ធ្វើការនៅថ្ងៃដំបូង \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>វាឈប់ធ្វើការនៅថ្ងៃដំបូង</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999504</td>\n","      <td>[-0.012111755087971687, -0.02565937303006649, ...</td>\n","      <td>[វាឈប់ធ្វើការនៅថ្ងៃដំបូង]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                  document  ...                   sentence\n","0  វាឈប់ធ្វើការនៅថ្ងៃដំបូង  ...  [វាឈប់ធ្វើការនៅថ្ងៃដំបូង]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":38}]},{"cell_type":"markdown","metadata":{"id":"lvE-LbNiPoBT"},"source":["# The Model understands Yiddish\n","![yi](https://www.worldometers.info/img/flags/small/tn_pl-flag.gif)"]},{"cell_type":"code","metadata":{"id":"sZlmLhajPoBb","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208309622,"user_tz":-300,"elapsed":769,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"b2b134e1-906c-4595-8015-c793c8132b77"},"source":["# Yiddish for: 'It was the best ever !!'\n","fitted_pipe.predict(\"עס איז געווען דער בעסטער טאָמיד !!\")\n","\t\t"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>עס איז געווען דער בעסטער טאָמיד !!</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.958143</td>\n","      <td>[0.01722853071987629, -0.04829197749495506, -0...</td>\n","      <td>[עס איז געווען דער בעסטער טאָמיד !!]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                             document  ...                              sentence\n","0  עס איז געווען דער בעסטער טאָמיד !!  ...  [עס איז געווען דער בעסטער טאָמיד !!]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":39}]},{"cell_type":"code","metadata":{"id":"5h-pha_nPoBc","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208310486,"user_tz":-300,"elapsed":885,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"c21dae04-2852-41d3-e0b9-0838ebbe5c7e"},"source":["# Yiddish for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"אויף דער ערשטער טאָג עס סטאַפּט ארבעטן \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>אויף דער ערשטער טאָג עס סטאַפּט ארבעטן</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.99991</td>\n","      <td>[-0.03324505686759949, -0.020611954852938652, ...</td>\n","      <td>[אויף דער ערשטער טאָג עס סטאַפּט ארבעטן]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                 document  ...                                  sentence\n","0  אויף דער ערשטער טאָג עס סטאַפּט ארבעטן  ...  [אויף דער ערשטער טאָג עס סטאַפּט ארבעטן]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":40}]},{"cell_type":"markdown","metadata":{"id":"XSz4WzScaAHj"},"source":["# The Model understands Kygrgyz\n","![ky](https://www.worldometers.info/img/flags/small/tn_kg-flag.gif)"]},{"cell_type":"code","metadata":{"id":"DXz6fhJSaAHu","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208310848,"user_tz":-300,"elapsed":369,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"e6a9ef09-b37d-4da8-e0ba-afb4cc8a8b17"},"source":["\t\t\n","# Kygrgyz for: 'It was the best ever !!'\n","fitted_pipe.predict(\"Бул эң мыкты болду !!\")\n","\t\t"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Бул эң мыкты болду !!</td>\n","      <td>average</td>\n","      <td>0</td>\n","      <td>0.989246</td>\n","      <td>[0.03657503426074982, -0.0562313050031662, -0....</td>\n","      <td>[Бул эң мыкты болду !!]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                document  ...                 sentence\n","0  Бул эң мыкты болду !!  ...  [Бул эң мыкты болду !!]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":41}]},{"cell_type":"code","metadata":{"id":"lh_ZSHlPaAHv","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208311821,"user_tz":-300,"elapsed":990,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"1ffda076-5f2c-4d2a-afa5-af7579807139"},"source":["\n","# Kygrgyz for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"Биринчи күнү эле иштебей калды \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>Биринчи күнү эле иштебей калды</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999909</td>\n","      <td>[0.012193242087960243, 0.014580711722373962, -...</td>\n","      <td>[Биринчи күнү эле иштебей калды]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                         document  ...                          sentence\n","0  Биринчи күнү эле иштебей калды  ...  [Биринчи күнү эле иштебей калды]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":42}]},{"cell_type":"markdown","metadata":{"id":"DGMVMKaTdJFj"},"source":["# The Model understands Tamil\n","![ta](https://www.worldometers.info/img/flags/small/tn_in-flag.gif)"]},{"cell_type":"code","metadata":{"id":"JWDr_LoCdJFn","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208312526,"user_tz":-300,"elapsed":716,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"645d21ae-34e6-4ff3-83f4-4d7a2c731370"},"source":["# Tamil for: 'It was the best ever !!'\n","fitted_pipe.predict(\"இது எப்போதும் சிறந்தது !! \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>இது எப்போதும் சிறந்தது !!</td>\n","      <td>good</td>\n","      <td>0</td>\n","      <td>0.829614</td>\n","      <td>[-0.03039463423192501, -0.058778341859579086, ...</td>\n","      <td>[இது எப்போதும் சிறந்தது !!]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                    document  ...                     sentence\n","0  இது எப்போதும் சிறந்தது !!  ...  [இது எப்போதும் சிறந்தது !!]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":43}]},{"cell_type":"code","metadata":{"id":"Q6C0BmTtdJFp","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620208312877,"user_tz":-300,"elapsed":366,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"0c8b2a5b-63e2-4088-ebe4-90550f71f9c8"},"source":["\t\t\n","# Tamil for: 'It stopped working on the first day'\n","fitted_pipe.predict(\"இது முதல் நாளில் வேலை செய்வதை நிறுத்தியது \")"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>document</th>\n","      <th>trained_classifier</th>\n","      <th>origin_index</th>\n","      <th>trained_classifier_confidence_confidence</th>\n","      <th>sentence_embedding_labse</th>\n","      <th>sentence</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>இது முதல் நாளில் வேலை செய்வதை நிறுத்தியது</td>\n","      <td>poor</td>\n","      <td>0</td>\n","      <td>0.999642</td>\n","      <td>[0.022033903747797012, -0.00905965268611908, 0...</td>\n","      <td>[இது முதல் நாளில் வேலை செய்வதை நிறுத்தியது]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                                    document  ...                                     sentence\n","0  இது முதல் நாளில் வேலை செய்வதை நிறுத்தியது  ...  [இது முதல் நாளில் வேலை செய்வதை நிறுத்தியது]\n","\n","[1 rows x 6 columns]"]},"metadata":{"tags":[]},"execution_count":44}]},{"cell_type":"markdown","metadata":{"id":"2BB-NwZUoHSe"},"source":["# 5. Lets save the model"]},{"cell_type":"code","metadata":{"id":"eLex095goHwm","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620209227950,"user_tz":-300,"elapsed":47226,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"68751c51-9c5a-403e-87f6-84fee139f186"},"source":["stored_model_path = './models/classifier_dl_trained' \n","fitted_pipe.save(stored_model_path)"],"execution_count":null,"outputs":[{"output_type":"stream","text":["Stored model in ./model/classifier_dl_trained\n"],"name":"stdout"}]},{"cell_type":"markdown","metadata":{"id":"e_b2DPd4rCiU"},"source":["# 6. Lets load the model from HDD.\n","This makes Offlien NLU usage possible!   \n","You need to call nlu.load(path=path_to_the_pipe) to load a model/pipeline from disk."]},{"cell_type":"code","metadata":{"id":"SO4uz45MoRgp","colab":{"base_uri":"https://localhost:8080/","height":80},"executionInfo":{"status":"ok","timestamp":1620209417898,"user_tz":-300,"elapsed":133849,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"ad55bee4-6794-4df2-bc97-a951f09688cc"},"source":["stored_model_path = './models/classifier_dl_trained'\n","hdd_pipe = nlu.load(path=stored_model_path)\n","preds = hdd_pipe.predict('It worked perfectly.')\n","preds"],"execution_count":null,"outputs":[{"output_type":"execute_result","data":{"text/html":["<div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>sentence_embedding_from_disk</th>\n","      <th>origin_index</th>\n","      <th>text</th>\n","      <th>from_disk_confidence_confidence</th>\n","      <th>sentence</th>\n","      <th>document</th>\n","      <th>from_disk</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>[[0.006914880592375994, -0.04569805786013603, ...</td>\n","      <td>8589934592</td>\n","      <td>It worked perfectly.</td>\n","      <td>[0.7951465]</td>\n","      <td>[It worked perfectly.]</td>\n","      <td>It worked perfectly.</td>\n","      <td>[average]</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>"],"text/plain":["                        sentence_embedding_from_disk  ...  from_disk\n","0  [[0.006914880592375994, -0.04569805786013603, ...  ...  [average]\n","\n","[1 rows x 7 columns]"]},"metadata":{"tags":[]},"execution_count":1}]},{"cell_type":"code","metadata":{"id":"e0CVlkk9v6Qi","colab":{"base_uri":"https://localhost:8080/"},"executionInfo":{"status":"ok","timestamp":1620209419447,"user_tz":-300,"elapsed":1537,"user":{"displayName":"ahmed lone","photoUrl":"","userId":"02458088882398909889"}},"outputId":"5ede73d9-5999-4b86-ebda-9f0464cedb00"},"source":["hdd_pipe.print_info()"],"execution_count":null,"outputs":[{"output_type":"stream","text":["The following parameters are configurable for this NLU pipeline (You can copy paste the examples) :\n",">>> pipe['document_assembler'] has settable params:\n","pipe['document_assembler'].setCleanupMode('shrink')                  | Info: possible values: disabled, inplace, inplace_full, shrink, shrink_full, each, each_full, delete_full | Currently set to : shrink\n",">>> pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'] has settable params:\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setExplodeSentences(False)  | Info: whether to explode each sentence into a different row, for better parallelization. Defaults to false. | Currently set to : False\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setStorageRef('SentenceDetectorDLModel_c83c27f46b97')  | Info: storage unique identifier | Currently set to : SentenceDetectorDLModel_c83c27f46b97\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setEncoder(com.johnsnowlabs.nlp.annotators.sentence_detector_dl.SentenceDetectorDLEncoder@5bb59739)  | Info: Data encoder | Currently set to : com.johnsnowlabs.nlp.annotators.sentence_detector_dl.SentenceDetectorDLEncoder@5bb59739\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setImpossiblePenultimates(['Bros', 'No', 'al', 'vs', 'etc', 'Fig', 'Dr', 'Prof', 'PhD', 'MD', 'Co', 'Corp', 'Inc', 'bros', 'VS', 'Vs', 'ETC', 'fig', 'dr', 'prof', 'PHD', 'phd', 'md', 'co', 'corp', 'inc', 'Jan', 'Feb', 'Mar', 'Apr', 'Jul', 'Aug', 'Sep', 'Sept', 'Oct', 'Nov', 'Dec', 'St', 'st', 'AM', 'PM', 'am', 'pm', 'e.g', 'f.e', 'i.e'])  | Info: Impossible penultimates | Currently set to : ['Bros', 'No', 'al', 'vs', 'etc', 'Fig', 'Dr', 'Prof', 'PhD', 'MD', 'Co', 'Corp', 'Inc', 'bros', 'VS', 'Vs', 'ETC', 'fig', 'dr', 'prof', 'PHD', 'phd', 'md', 'co', 'corp', 'inc', 'Jan', 'Feb', 'Mar', 'Apr', 'Jul', 'Aug', 'Sep', 'Sept', 'Oct', 'Nov', 'Dec', 'St', 'st', 'AM', 'PM', 'am', 'pm', 'e.g', 'f.e', 'i.e']\n","pipe['sentence_detector@SentenceDetectorDLModel_c83c27f46b97'].setModelArchitecture('cnn')  | Info: Model architecture (CNN) | Currently set to : cnn\n",">>> pipe['bert_sentence@labse'] has settable params:\n","pipe['bert_sentence@labse'].setBatchSize(8)                          | Info: Size of every batch | Currently set to : 8\n","pipe['bert_sentence@labse'].setCaseSensitive(False)                  | Info: whether to ignore case in tokens for embeddings matching | Currently set to : False\n","pipe['bert_sentence@labse'].setDimension(768)                        | Info: Number of embedding dimensions | Currently set to : 768\n","pipe['bert_sentence@labse'].setMaxSentenceLength(128)                | Info: Max sentence length to process | Currently set to : 128\n","pipe['bert_sentence@labse'].setIsLong(False)                         | Info: Use Long type instead of Int type for inputs buffer - Some Bert models require Long instead of Int. | Currently set to : False\n","pipe['bert_sentence@labse'].setStorageRef('labse')                   | Info: unique reference name for identification | Currently set to : labse\n",">>> pipe['classifier_dl@labse'] has settable params:\n","pipe['classifier_dl@labse'].setClasses(['average', 'poor', 'good'])  | Info: get the tags used to trained this ClassifierDLModel | Currently set to : ['average', 'poor', 'good']\n","pipe['classifier_dl@labse'].setStorageRef('labse')                   | Info: unique reference name for identification | Currently set to : labse\n"],"name":"stdout"}]},{"cell_type":"code","metadata":{"id":"EY3jdCj41bJC"},"source":[""],"execution_count":null,"outputs":[]}]}